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ABSTRACT 



The "von Neumann bottleneck" imposes severe limitations 
on programming languages. This thesis points out that 
although the hardware limitations imposed by this bottleneck 
are being overcome, its constraints will remain in programs 
as long as there are assignment statements in their code. 
We assert that functional programming languages allow us to 
harness the processing power of computers with hundreds or 
even thousands of processors, and also allow is to soxve 
problems which are time/cost prohibitive on a uniprocessor. 

We discuss a mechanical method for transforming impera- 
tive programs into functional programs. We feal that the 
mechanical transformation process is very inexpensive, and 
that it might be the best way to make imperative "library" 
programs into functional ones which are well suited to 
concurrent processing. 
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I. HISTORY AND INTRODUCTION 



The von Neumann architecture was a brilliant break- 
through in the development of computers. Through this design 
computing machines achieved an execution speed and power 
which was foreseen only by men of great vision. 

Unfortunately, word-at-a-time processing, which is implicit 
in this architecture, has become a limiting factor in the 
advancement of machine speed. 

The so-called von Neumann bottleneck can be overcome in 
computer architecture. Indeed, there are many architectures 
which employ a variety of technigues to circumvent the 
bottleneck, by using multiple buses along with multiple 
central processing units (CPUs). For many decades, commer- 
cial computers have been structured to handle information 
sequentially. Now, scientists are trying to replace the 
large computer, based on serial instructions, with networks 
of small computers linked in a way that would enable them to 
work on different parts of a problem concurrently [Ref. 1 ]. 
Many experts forecast that Japan’s fifth generation computer 
systems will make the Smithsonian Institute the only appro- 
priate place in which to house von Neumann machines. These 
new computers virtually eliminate von Neumann bottlenecks 
[Ref. 2]. 

Not so obvious is the fact that the von Neumann bottle- 
neck has become manifest not only in computer architecture, 
but in the languages which were designed with these machines 
in mind. Since the development of Fortran ia the early 
1950 ’s, high-level programming languages have been based in 
large part on the instruction sets of their "target 
machines". Fortran was a very efficient language, and it 
achieved that efficiency because its optimizer was developed 
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with the instruction set of the IB II 731 in the forefront of 
the designer’s mind [Ref. 3: p.33]. Since that time, the 
von Neumann bottleneck has firmly astablishei itself in 
every imperative programming language. The bottleneck is 
manifested in the form of assignment statements [Ref. 4]. 
He thus find ourselves in a situation where the high-level 
languages we ordinarily use are not capable of taking advan- 
tage of the computing power of state-of-the-art machines. It 
seems obvious that computing power which cannot oe harnessed 
is not of much value to us. What can we do about that? This 
is the question which provided tne motivation for this 
thesis. 

Since high-level languages have oeen to a very great 
extent designed with the instruction sets of their target 
machines in mind, there are limitations built into the 
structure of the languages which will be very difficult to 
overcome. I will spend some time focusing on the weaknesses 
of the imperative languages. I would like to say at the 
outset, however, that I in no way mean to imply that impera- 
tive languages are not extremely useful in many applica- 
tions. The limitations on which I will primarily focus will 
be in terms of imperative languages as applied to co nc u rren t 
processing . 

Similarly, I will , discuss functional programming 
languages. I believe that they provide some relief from the 
von Neumann bottleneck that cannot be achieved by working 
within the framework of imperative languages. Put another 
way, I believe that functional programming languages will 
enable the user to take advantage of the power afforded by 
these new multiprocessor machines in a way that imperative 
languages simply cannot. Indeed there are techniques which 
can be employed which will extend the "concurrent processing 
power" of imperative languages, but these techniques will 
never manifest all the advantages that functional languages 
afford, such as evaluation order independence. 
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There are techniques available which allow for impera- 
tive programs to be translated into functional oies [Ref. 5: 
pp. 136-149]. I will demonstrate one of these techniques on 
a widely used imperative program: the Shell sort. It should 
be noted, however, that functional programs also have their 
limitations; but these limitations seem to appLy to areas 
other than the concurrent processing issue. 

Aside from the fact that mult iprocessor aardware is 
becoming available, there is another important reason for 
wanting to develop and exploit the properties of functional 
programming languages which enhance concurrent processing. 
Research conducted for NASA by an independent research firm 
[Ref. 6] discusses a whole class of problems that are today 
too computationally complex to be accomplished using conven- 
tional computer resources. For example the linear static 
analysis of an undersea oil platform was conducted using 
finite-element structural analysis. The problem had over 
720,000 degrees of freedom, and took about one week of 
processing time on a Uni vac 1110 computer [Ref. 5: pp. 7-8]. 
The same authors point out that in the data-flow machine 

operators "fire” as soon as their operands are available. 

This is exactly how functional programs work: a function 

"fires” as soon as all of its parameters are available! 

Although programs expressed in sequential languages have 

been successful at expressing parallelism to some degree, 
they do not appear to have the potential of detecting paral- 
lelism of a high degree (100 or more processors) [Ref. 6: 

p. 20 ]. 

As seems so often to be the case in computer science 
issues, no one technique will serve as a panacea. Functional 
programming is no exception. Rather, it provides the user 
with a great number of advantages, particularly in the area 
of concurrent processing. These must be weighed against the 
disadvantages, and a decision can then be made uased on the 
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specific application for which the program is intended. I 
hope that this paper will help provide the background for 
that decision-making process. 
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II. IMPERATIVE LANGUAGE S: strengths and limitations 

A. CONVERSATIONS with machines 

Computers , if properly directed, have the ability to 
execute a great many instructions in a relatively short 
period of time. let in order to harness this computational 
power, one must be able to communicate with the computer, 
and give it some "marching orders". For quite some time, 
the only way in which to effectively communicate with 
computing machines was to use the machine’s native language 
(cleverly dubbed "machine language"). Indeed, many people 
learned to use machine language very well, ani some even 
began to l ike it!!! To most people, however, talking to a 
machine was quite a strange concept. Talking using an 
alphabet consisting of only zeros and ones was even more 
bizarre! There seemed to be two camps which developed from 
this "language problem". One camp lived for computers. They 
were convinced that the future of the world belonged to 
those who could "speak" machine language, ani spared no 
effort in becoming friends with their inanimate associates. 
The other camp was at the same time enamored with, skeptical 
of, and intimidated by these new machines. These people 
swore that' the slide rule would never be replaced, and that 
the computers were more trouble than they were worth. 

To some extent, both camps were right. Computers 
certainly do have the the ability to complete tedious, 
boring tasks very quickly and very accurately. Even today, 
however, it seems that it can sometimes be more trouble than 
it is worth to get the machine to do what we want. In fact, 
sometimes it seems that we are wocking for the computer, 
instead of the computer serving us as it should be. The 
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development of high-level languages was undertaken in large 
part to narrow the gap between the two camps described 
above . 



B. ADVANTAGES OF HIGH-LEVE 
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user would be able to use a natural (spoken) language, and 
the system would take :are of converting taat to the 
language of the target machine. 

Although this goal seems unachieveable , it Is certainly 
something for which we should strive. We should make every 
effort tc make programming languages understandable (to the 
human), and at the same time keep error-checkin g features, 
such as strong typing, embedded in them. 

High-level programming languages free the user from some 
of the details of machine implementation, ana hence these 
languages are more powerful and understandaole than machine 
languages. Figure 2.1 illustrates a simple "add" instruc- 
tion written in four ways: machine language [Ref. 10], 

assembly language [Ref. 10], high-level language, and 
natural language. 

Another advantage of high-level languages is that they 
are transporta bl e. i.e. , they can be used on more than one 

type (brand/model) of machine. Compilers and interpreters 
take care of translating them into the instruction set of 
the target machine. Programs written in high-level languages 
are therefore easier to maintain throughout their life 
cycle . 

Through the years, high-level languages have become more 
powerful and more understandable. In the next section I will 
discuss the evolution of high-level languages. 

C. THE EVOLOTION OF IMPERATIVE LANGUAGES 

Kith Figure 2.1 in mind, 1 it’s hard to imagine how 
people put up with machine language for so long! As we shall 
see, successive generations of high-level languages have 



1 Note that Figure 2.1 is an extremely simple example. 
When conditional expressions, looping, and recursion are 
introduced. the differences in complexity among the 
different types of languages become even more pronounced. 
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stop program 



High Level Language 

Eegin 
A := 25; 

E := 7; 

Sum : = A + B; 
WRITE LN (Sum) 

End. 



N atu r al Lan gu age 
Print the sum of 25 and 7. 



mulator 

n) 



Figure 2.1 Language Comparisons for a SimpLe "Add" 

made programming much easier, but many feel it is still too 
complex and tedious for the average user to pick up. Thus 
the ultimate users of computing power--b usinessme n , accoun- 
tants, scientists and engineers — still require a middleman 
to communicate with their machines [Ref. 12]. 

As we quickly look at the development of imperative 
programming languages, let’s keep in mind the attributes 
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